The principles of science-based medicine include using all available scientific evidence to assess the safety, efficacy, and risk vs benefit of any medical intervention. Further, treatments supported by rigorous evidence should be preferred over those with less scientific support, and there should be a lower boundary of plausibility and evidence below which it is unethical to recommend treatments. Finally, the ethics of patient autonomy require transparency and honesty. Patients cannot make informed decisions and cannot protect their autonomy if they are lied to.
There are regulations about what kinds of claims can be made for medical products and services, but they vary from country to country, and generally are fairly loose. Even when regulations exist, enforcement is largely inadequate. This leaves consumers fending for themselves when it comes to important medical decisions involving complex and technical scientific data.
This is an endemic and significant problem in the arena of “alternative” medicine. This artificial category was created in order to market dubious treatments that lack scientific support. Proponents have been working for decades to weaken standards of care for themselves and their treatments, change the rules of science, and carve out a favorable regulatory space in which to operate. When you lower standards in this way it should be no surprise that all standards suffer – including basic transparency and honesty.
A review of acupuncture claims
A recent study published in the New Zealand Medical Journal reviews claims made for acupuncture on practitioner websites. They found:
A search of acupuncturists’ websites shows that many claim to be able to treat a wide range of conditions, despite a lack of evidence showing the efficacy of acupuncture in the treatment of those conditions. Practitioners and owners of websites likely to be in breach of the Medicines Act include many committee members from acupuncture’s professional bodies.
The study was performed by Daniel Ryan from the Society for Science Based Healthcare. He compared the claims to a list of medical conditions that it is illegal to claim to treat without approval, and found:
… for the 101 included websites, the three most frequent claims were around the treatment or prevention of mental illness, infertility and arthritis (all on the list).
Combined, these claims appeared on 73 per cent of the websites. He also found 11 per cent of sites claimed acupuncture could treat or prevent cancer, 23 per cent made diabetes claims, 19 per cent targeted thrombosis and 14 per cent offered help for heart disease.
For comparison, I did a quick survey of 10 acupuncturist websites local to me in Connecticut. Two sites did not make specific disease claims, only offered acupuncture for a “variety of conditions.” The other eight sites made specific claims to be able to treat many conditions, including arthritis, hypertension, migraine, Lyme disease, anxiety, allergies, insomnia, Bell’s palsy, stroke symptoms, and more. Half of the sites claimed to treat over 50 conditions with acupuncture. None claimed to treat cancer, but half claimed to treat the side effects of chemotherapy or cancer pain.
Kate Roberts from Acupuncture New Zealand, in response to Ryan’s article, said in an interview:
Roberts said Ryan was also a member of the Skeptics Society and the report was part of a “witch hunt” against acupuncture that was “tiresome, often misinformed and misleading”.
“They say acupuncture has no evidence base, but that’s absolutely incorrect. There are huge numbers of systematic reviews on acupuncture proving that it is a good treatment for many conditions and that is why it is in 1000 clinical guidelines worldwide.”
That, of course, is the question – what does the evidence say about the effectiveness of acupuncture?
Acupuncture does not work
A science-based review of acupuncture finds that the claims made for it are highly implausible and there is no single indication for which there is robust evidence of efficacy. Yet proponents, like Roberts, claim that there is evidence. Why the disconnect?
I won’t speculate about the honesty of proponents like Roberts – I don’t know what she really believes, but that does not really matter. It is possible that she simply has a very different idea of what constitutes scientific evidence than I do. I my experience when a proponent of alternative medicine claims that a treatment is evidence-based or backed by science they mean that there is some study somewhere that was positive. They do not mean that the evidence meets a reasonable science-based standard.
To review that standard (which bears frequent repeating):
- Claims should be clearly defined. For example, with acupuncture, proponents need to define exactly what it is. There is a consistent definition of acupuncture on the sites I reviewed and in general – sticking thin acupuncture needles into specific acupuncture points. However, acupuncturists cannot change the definition as needed when the evidence is negative (by counting poking the skin in random locations as acupuncture, or electrical stimulation as acupuncture).
- There should be rigorous trials of sufficient power that are properly blinded with a reasonable assessment of the blinding, showing clinically and statistically significant positive results.
- Independent replication showing an overall pattern of effectiveness in the literature.
- The clinical evidence needs to be proportional to the scientific plausibility of the claims being made.
Even if we do not count the fourth criterion regarding plausibility, acupuncture does not meet these basic standards for being science based for any indication. Acupuncture studies generally are of poor quality. High quality well-controlled studies generally are negative.
So how can Roberts claim that there are systematic reviews showing acupuncture works? There are several possible reasons for this. First – she is simply wrong. She is mischaracterizing the medical literature.
We have published many reviews of acupuncture systematic reviews over the years on SBM – all negative. But just to take a fresh look at the literature, I searched for “acupuncture systematic reviews” in Pubmed to see what came up.
The first was a systematic review of various treatments for fatigue from cancer and in hematopoietic stem cell transplant recipients – no benefit from acupuncture.
The second was acupuncture for neuropathic pain, concluding:
Due to the limited data available, there is insufficient evidence to support or refute the use of acupuncture for neuropathic pain in general, or for any specific neuropathic pain condition when compared with sham acupuncture or other active therapies.
The third was for chronic knee arthritis, which found that while there may be a benefit:
However, given the heterogeneity and methodological limitations of the included trials, we are currently unable to draw any strong conclusions regarding the effectiveness of acupuncture for chronic knee pain.
Next was endometriosis-related pain, finding:
Of the 10 studies included, only one pilot study used a placebo control and assessed blinding
Laser acupuncture for obesity:
Although some studies have indicated beneficial effects for LA on obesity, the lack of evidence with high methodological quality made it impossible to reach a definitive conclusion about the efficacy of LA for obesity management.
These were just the first reviews to come up, and so is not a thorough or systematic assessment. However, this is generally what we find – there may be some positive evidence, but we can’t really say, the study quality is too low, etc. When we do get really rigorous clinical trials, they generally show that acupuncture does not work – it does not matter where you stick the needles, if they even penetrate the skin, or if you even use needles at all.
So part of the problem comes from proponents interpreting weak preliminary and flawed evidence as if it is definitive, when it is not even adequate (especially given the low scientific plausibility of acupuncture).
However, the problems go deeper than that. The evidence base itself, including the systematic reviews, are rigged in favor of acupuncture. Edzard Ernst, who has published many reviews of acupuncture, lays out the problems:
First, all acupuncture studies coming out of China are positive. Even if acupuncture worked, this should not be true. This unequivocally indicates massive bias in the Chinese acupuncture literature. This further means that any acupuncture systematic review that includes studies from China are contaminated with unreliable, and positively biased, data.
He also points out that many reviews include Chinese studies that are only available in Chinese, which means many reviewers will not be able to read the original articles. Further, many included studies have fatal methodological flaws. He concludes:
Given this situation, the conclusions of any review for which there are only Chinese acupuncture studies might as well be written before the actual research has started. If the authors are pro-acupuncture, as the ones of the present article clearly are, they will conclude that “acupuncture is efficacious”. If the research team has some critical thinkers on board, the same evidence will lead to an entirely different conclusion, such as “due to the lack of rigorous trials, the evidence is less than compelling.”
That right there is the difference between the conclusion of proponents and the conclusion of advocates of science-based medicine.
But there are some other problems that Ernst did not mention, at least not in that one article. A recent review of registered clinical trials of Traditional Chinese Medicine (54.4% of which are acupuncture trials) found that only 8.7% of completed trials had reported their results. This survey was done two or more years after study completion – that is an extremely low publication rate. The implication is that there may be massive publication bias in the acupuncture literature, with negative studies less likely to see the light of day.
Also, as I mentioned, many systematic reviews include things like electroacupuncture, which muddies the waters by including electrical stimulation, which may have an effect that has nothing to do with acupuncture.
Further, it is difficult to properly blind acupuncture studies because it is an invasive treatment. Studies should assess the effectiveness of the blinding in the study, and if they don’t that calls into question the results. This is especially true when following a subjective outcome and an invasive treatment.
Even with all these factors, essentially rigging the game in favor of acupuncture, systematic reviews do not show that acupuncture works for anything. It is not uncommon for reviewers who are pro-acupuncture to try to spin their conclusion to be as positive as possible, as Ernst points out, but the actual results do not support the conclusion that acupuncture works.
We are left with a situation in which acupuncture proponents claim that acupuncture works for a long list of medical indications, and claiming that this is supported by evidence. Meanwhile the actual evidence, fairly and rigorously evaluated, is negative.
